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COLOR IMAGE PICKUP APPARATUS As a result, pixel signals (Mg+Cy) and (Ye and Gr), and 

HAVING A PLURALITY OF COLOR FILTERS (Cy+Gr) and (Mg+Ye) are alternately output from the image 

pickup element For the sake of convenience, these four 

This application is a division of allowed application Ser. signals are respectively represented by 

No. 08/133,214, filed Oct 7, 1993, U.S. Pat No. 5,541,648. 5 C^Mg+Cy 

BACKGROUND OF THE INVENTION C^Ye+Gr 

1. Field of the Invention CarCy+Gr 

The present invention relates to a color image pickup ? 4 ^^ + ^ c 

apparatus having an image pickup element constituted by a 10 ^ this case, when these four signals arc obtained, a group 

plurality of two-dimendonally arranged pixels. of s P atial ^^Jcenta positions can be considered as the 

n r> w a Ti i a a sampling structure shown in FIG. 3, which is based on the 

plated Background Art signals C„ Cj, and C„. Therefore, in the mixed pixel 

With the recent advances in the semiconductor read mode> ^ ^ ^ ^ a fa mG& MA 

dJgtal signal processing has become a mam ^ ^ arc to an image pickup element having 

tendinthefleldofvidcosigndra^ssing.Tnis^because a color filter arrangement k which color filters of different 

digital signal processing is more advantageous than analog ^ m tedl ed at a te ^ ^ u me 

agnalprocessingu i terms of the integration and definition of horizontal direction. A color image is obtoedbyforrning a 

JO, the easiness of various adjustments, S/N ratio, and the luminance dgnal ^ a colar rigIlal for ^ field on mc 

c ' 20 b as * s °f tncsc P* 0 * signals. 

For this reason, image pickup elements of various However, when the image pickup elements having the 
schemes adapted for digital signal processing have been color filter arrangements shown in FIGS. 1 and 3 arc to be 
developed because they have the advantages of small size, use d t in order to extract luminance and color signals, alter- 
light weight, high reliability, and the like. Of these image natc ^ ^ (c g t ^ third line data in an odd field, 
pickup elements, CCD type image pickup elements having second and fourth line data in an even field) must be 
the color filter arrangements shown in FIGS. 1 and 2 are usec ^ t 

most widely used nowadays as image pickup elements for with this arrangement, since data at two points spatially 

video cameras and electronic still cameras in consideration separated from each other in the vertical direction are added 

of resolution, S/N ratio, easiness of signal processing, and together, the correlation in the vertical direction deteriorates. 

tneu ^ £B - ... 30 As a result, the resolution in the vertical direction decreases, 

When frame photography of a still image is to be per- and the generation amount of pseudo color (color moire) 

formed by using an electronic still camera or the like, an increases. 

image pickup element having the color filter arrangement Especially when photography of a still image is to be 

shown in FIG. 1, in which color filters of different colors are performed by using an electronic stfll camera, since only an 

repeatedly arranged at a two-pixel period in the horizontal 35 image of one field can be obtained by the mixed pixel read 

direction, must be used for the following reason. A pixel method, a high-resolution image, i.e., a high-resolution 

signal formed by the image pickup element is read out frame image, cannot be obtained, 

alternately from an odd field and an even field, and identical in general, the generation of this color moire" can be 

outputs must be obtained from the two fields. suppressed by an optical low-pass filter placed in front of the 

Referring to FIG. 1, the image pickup element has a pitch ^ image pickup element On the other hand, the use of this 

Fh in the horizontal direction and a pitch Pv in the vertical optical low-pass filter leads to a decrease in the resolution in 

direction. the horizontal direction. Far this reason, in a conventional 

The image pickup element shown in FIG. 2 has a color video camera, an optical low-pass filter must be designed to 

filter arrangement different from the arrangement in which determine image quality in consideration of the trade-off 

color filters of different colors are repeatedly arranged at a 45 between the resolution and the color moire" caused by color 

two-pixel period in the horizontal direction. carriers. 

This image pickup element has a color filter arrangement Recently, in order to improve image quality, a two-plate 

in which color filters of different colors are repeatedly scheme color image pickup apparatus using two image 

arranged at a two-pixel period in the horizontal direction. pickup elements and a single plate scheme color image 

This filter arrangement is constituted by lines on which 50 pickup apparatus using an image pickup element designed to 

magenta (Mg) and green (Gr) filters are alternately arranged, ensure image quality as high as that of the two-plate scheme 

and lines on which cyan (Cy) and yellow (Ye) filters are color image pickup apparatus have been used. Even these 

alternately arranged. The Mg and Gr filters are arranged in color image pickup apparatuses are not free from the above- 

the form of an offset sampling array, whereas the Cy and Ye described problems, i.e., the occurrence of color moire' and 

filters are arranged in the form of a square sampling array. 55 the inability to obtain sufficient resolution. 

Note that some image pickup element has a color filter qttmiuarv of thp Twvrwrrnw 

arrangement in which Mg and Gr filters are arranged in the SUMMARY OF THE INVENTION 

form of a square sampling array, and Cy and Ye filters are The present invention has been made in consideration of 

arranged in the form of an offset sampling array. the above situation, and has as its object to provide a color 

When signals are to be read out from the image pickup 60 image pickup apparatus which has high resolution and a 

element shown in FIG. 2, a method called a mixed pixel read small amount of color moire, and which can photograph a 

(or a field read) method is generally used. In this method, still frame image. 

photoelectricalry converted signal charges are read out, two According to an embodiment of the present invention, 

lines at a time, in a mixed state. According to the method, there is provided a color image pickup apparatus comprising 

signal charges are read out in the order of A^_ l5 A*, A^ 1} . 63 an image pickup element having four types of color filters 

, . in die A field, and in the order of B^, B n , B^, ... in which are arranged to form an offset sampling structure in 

the B field. which color filters of different colors are repeatedly arranged 
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at a two-pixel period in a horizontal direction, and color 
filters of different colors are repeatedly arranged at a two- 
pixel period in a vertical direction while being offset in the 
horizontal direction by one pixel, an optical low-pass filter 
having an optical member for splitting an incident light 5 
beam into two light beams separated from each other by a 
distance D in a direction inclined clockwise or counterclock- 
wise at an angle 6 with respect to a scanning direction of the 
image pickup element, the distance D satisfying conditions 
defined by 10 

1.6 ?WWA sin 6+2 Pv 00s 8gX>£2.4 PhPvAPh sm 0+2 Pv cos 
ei(0SeSft/2) 

where Ph is a pitch of the image pickup element in the 
horizontal direction, and Pv is a pitch of the image pickup 
element in the vertical direction, and a memory far tempo- 15 
rarily storing an output from the image pickup element, 
wherein necessary information is read out from the memory, 
and digital signal processing is performed to obtain an 
image. 

According to another embodiment of the present 20 
invention, the apparatus further comprises gain adjustment 
means for adjusting a gain of an output from the image 
pickup element to make outputs from the four types of color 
filters constant with respect to an achromatic object, and 
signal formation means for forming R, G, and B signals from 25 
the outputs from the four types of color filters by perf arming 
a matrix operation using a 3x4 matrix designed such that a 
sum of coefficients of two arbitrary columns in every row is 
equal to a sum of coefficients of remaining two columns. 

According to still another embodiment of the present 30 
invention, there is provided a color image pickup apparatus 
comprising: 

an image pickup element having four types of color filters 
which are arranged to form an offset sampling structure in 
which color filters of different colors are repeatedly arranged 35 
at a two-pixel period in a horizontal direction, and color 
filters of different colors are repeatedly arranged at a two- 
pixel period in a vertical direction while being offset in the 
horizontal direction by one pixel, and signal formation 
means far forming R, G, and B signals from the outputs from 40 
the four types of color filters by performing a matrix 
operation using a 3x4 matrix designed such that a sum of 
coefficients of two arbitrary columns in every row is equal 
to a sum of coefficients of remaining two columns. 

According to still another embodiment of the present 45 
invention, there is provided a color image pickup apparatus 
comprising an image pickup element having a plurality of 
pixels two-dimensionally arranged in horizontal and vertical 
directions, wherein horizontal lines twice as many as the 
number of horizontal lines required to obtain a desired 50 
image are arranged in the image pickup element, the hori- 
zontal lines are first and second horizontal lines which are 
alternately arranged such that first color filters are arranged 
on each first horizontal line and second and third color filters 
are arranged for every other pixels on each second horizon- 55 
tal line, a first difference signal is formed by calculating a 
difference between a signal output from a given pixel for 
which the second color filter is arranged and a signal output 
from a pixel which is on the same column as that of the given 
pixel and for which the first color filter is arranged, a second 60 
difference signal is formed by calculating a difference 
between a signal output from a given pixel for which the 
third color filter is arranged and a signal from a pixel which 
is on the same column as that of the given pixel and for 
which the first color filter is arranged, and color difference 65 
signals are formed on the basis of the first and second 
difference signals. 



017 

4 

According to still another embodiment of the present 
invention, there is provided a color image pickup apparatus 
comprising a first image pickup element having first color 
filters arranged therein, and a second image pickup element 
having second and third color filters arranged therein, the 
first and second image pickup elements being used to obtain 
a desired image, wherein a first difference signal is formed 
by calculating a difference between a signal output from a 
given pixel, in the' second image pickup element, for which 
the second color filter is arranged and a signal output from 
a pixel, in the first image pickup element, which is located 
at a position equivalent to that of the given pixel on an 
imaging plane, and for which the first color filter is arranged, 
a second difference signal is formed by calculating a differ- 
ence between a signal output from a given pixel, in the 
second image pickup element, for which the third color filter 
is arranged and a signal output from a pixel, in the first 
image pickup element, which is located at a position equiva- 
lent to that of the given pixel on an imaging plane, and for 
which the first color filter is arranged, and color difference 
signals are formed on the basis of the first and second 
difference signals. 

In this case, a luminance signal is formed by synthesizing 
a signal output from a pixel for which the first color filter is 
arranged and the first and second difference signals. 

Alternatively, a luminance signal may be formed on the 
basis of a signal output from a pixel for which the first color 
filter for a luminance signal is arranged. 

According to the above-described embodiments, four 
types of color niters are arranged in an image pickup 
element to form an offset sampling structure in which the 
color filters are repeatedly arranged at 2 Ph in the horizontal 
direction, and the color filters are repeatedly arranged at 2 Pv 
in the vertical direction while being offset by Ph in the 
horizontal direction. With this arrangement, the color carri- 
ers generated in the horizontal and vertical directions can be 
reduced, and hence color moire* caused by the color carriers 
in the vertical and horizontal directions can be reduced, 
thereby increasing the resolution to the resolution limit 

In addition, satisfactory MTF characteristics can be 
obtained in a region lower than the resolution limit, because 
an optical low-pass filter is constituted by an optical member 
for splitting an incident light beam into two light beams 
separated from each other by a distance D satisfying pre- 
determined conditions in a direction inclined clockwise or 
counterclockwise at an angle 0 with respect to the scanning 
direction of the image pickup element Therefore, the reso- 
lution can be increased to the resolution limit In addition, 
since frequency components at frequencies higher than the 
frequency of the resolution limit are sufficiently suppressed, 
color moire' caused by aliasing distortion of color carriers 
can be reduced. 

The gain of an output from the image picknp element is 
adjusted to make outputs from the four types of color filters 
constant In addition, a matrix operation to be performed 
when the R, G, and B signals are obtained from these outputs 
is performed by using a 3x4 matrix which is designed such 
that the sum of the coefficients of two arbitrary columns of 
every row is equal to the sum of the coefficients of the 
remaining two columns. With this operation, the color 
carriers generated in the vertical direction can be further 
reduced, and hence color moir6 in the vertical direction can 
be further reduced. 

If the color image pickup apparatus of the present inven- 
tion is a multiple-plate scheme color image pickup apparatus 
using a plurality of image pickup elements or a single-plate 
scheme color image pickup apparatus using an image pickup 
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element having horizontal lines twice as many as those of a FIG. 13 is a graph showing the MTF characteristics of the 

general image pickup element, first to third color filters of optical low-pass filter according to the present invention; 

different colors are arranged fox the respective pixels. With FIG. 14 is a graph showing the frequency characteristics 

this arrangement, two types of difference signals are formed of the optical low-pass filter according to the present inven* 

by calculating the differences between signals obtained from s tion; 

given pixels for which the second and third color filters are HG. 15 is a block diagram showing a color image pickup 

respectively arranged, and signals obtained from pixels apparatus according to the third embodiment of the present 

which are at positions equivalent to those of the given pixels invention; 

on the imaging plane and for which the first color filters are FIG, 16 is a block diagram showing the schematic 

arranged. Color difference signals aTe obtained on the basis 10 arrangement of a conventional two-plate scheme color 

of these difference signals. Therefore, the color carriers image pickup apparatus; 

generated in the horizontal and vertical directions can be 17A ^ 17B ^ e charts showing the filter arrange- 

reduced, thus reducing color moir6 in the horizontal and ment8 ^ pi^p elements used for the conventional 

vertical directions and increasing the resolution to the reso- two-plate scheme color image pickup apparatus; 

nation i limit 15 n(J 18 is ft ^Q^g ±c ^ arrangement of an 

- 10 , S image pickup element used for a single-plate scheme color 

^tT^nn 1 ^ ^ a PP™ **** *> ^ quality 
for which thefirst color filter is arranged. Correction pro- cquivalcnt t0 that obtained by the conventional two-plate 
cessing is Performed in this manner ; Jo obtain ^ a proper scheme ^ ^ ^ cku ^ ^ 
luminance signal matching visual sensitivity. In addition, as 20 -™ 7\ ft T» ~L u • . 
the first color filter, a color lUter having spectral character- J 10 *' ? f U * « Aowm ^ frequency Chr- 
isties matching visual sensitivity may be used. If a lumi- !?f$L s ba f ' ° n me ^ *h°wn » 
nance signal is formed on the basis of a signal obtained from ', an M \ J . t . , . 
a pixel for which this first color filter is arranged, the FIG. 20 is a block diagram showing a color image pickup 
processing can be simplified. 25 fl^a^ according to the fourth embodiment of the present 

The above and other objects, features, and advantages of invention; 

the present invention will be apparent from the following FIG ' " a lowing the filter arrangement of an 

detailed description in conjunction with the accompanying image pickup element used for the fourth embodiment; 

drawings and the appended claims. HG* 22 is a Hock diagram showing a color image pickup 

30 apparatus according to the fifth embodiment of the present 

BRIEF DESCRIPTION OF THE DRAWINGS invention; 

FIG. 1 is a chart showing the filter arrangement of an , FIG. 23 is a chart showing the filter arrangement of an 

image pickup clement used for a conventional color image P icku P aemeut "»d for mc ™ embodiment; 

pickup apparatus; FIG- 24 is a block diagram showing a color image pickup 

FIG. 2 is a chart showing the filter arrangement of an 35 according to the sixth embodiment of the present 

image pickup element used for a conventional color image invention, 

pickup apparatus; FIGS. 25A and 25B are charts showing the filter arrange- 

f-rr/i *% • . \ « . , . j . ments of image pickup elements used for the sixth embodi- 

FIG. 3 is a chart showing a sampling structure used when men t: 

a mixed pixel read operation is performed in the image ^ _Jl . ... . . . . . 

pickup dement shown to FIG. 2; 40 nG L 26 18 a W°ck diagram showing a color image pickup 

t^A - . . , , j. . , , , . , apparatus according to the seventh embodiment of the 

FIG. 4 is a block diagram showing a color image pickup present invention; 

apparams according to the first embodiment of the present nQS „ Aaai27B m charts showing ^^^^ 

^„ tl , t . ' . . , ments of image pickup elements used for the seventh 

FIG. 5 is a block diagram showing a color image pickup 45 einbodiment- 

TEX 10 ^ ^» d Cldboah ™ nt ° f &e FIGS. 2*U and 28B are charts showing the filter arrange- 

m ^ en °f' . . „ meats of image pickup elements used for the seventh 

FIG. 6 is a chart showing the filter arrangement of an embodiment* 

image pickup element used for the first and second embodi- nGS ^ ^ M m showlng me f^ ency 

' characteristics of the image pickup elements used for the 

FIG. 7 is a chart showing the filter arrangement of an seventh embodiment- 
image pickup element used for the first and second embodi- nGS and 30B are charts showing the filter arrange- 
mcnts » ments of image pickup elements used for the seventh 

HG. 8 is a graph showing the frequency characteristics of embodiment; 

the image pickup element used in the first and second 55 HGS> 31A arid 31B are graphs showing the frequency 

embodiments; characteristics of the image pickup elements used for the 

FIG. 9 is a graph showing the frequency characteristics of seventh embodiment; 

the image pickup element used in the first and second FIG. 32 is a block diagram showing a color image pickup 

embodiments; ^ apparatus according to the eighth embodiment of the present 

FIG. 10 is a graph showing the frequency characteristics invention; 

of the image pickup element used in the first and second FIGS. 33A and 33B are charts showing the filter arrange- 

embodiments ; ments of image pickup elements used for the eighth embodi- 

FIG. 11 is a view showing the arrangement of an optical ment; 

low-pass filter according to the present invention; 63 FIG. 34 is a chart showing still another embodiment of the 

FIG. 12 is a chart showing optical paths formed by the color filter array of the image pickup element according to 

optical low-pass filter according to the present invention; the present invention; 
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FIG. 35 is a graph showing the characteristics of the 
sampling structure of the image pickup element shown in 
FIG. 34 in a frequency space; 

FIG. 36 is a block diagram showing an embodiment of the 
present invention; 

FIG. 37 is a block diagram showing another embodiment 
of the present invention; 

FIG. 38 is a block diagram showing still another embodi- 
ment of the present invention; 

FIG. 39 is a chart showing a method of reading out signals 
from the color filter array of an image pickup element; 

FIG. 40 is a chart showing a method of inputting image 
signals read out in FIG. 39 to a primary color separation 
matrix; 

FIG. 41 is a block diagram showing still another embodi- 
ment of the present invention; 

FIG. 42 is a chart showing a color filter pattern according 
to the above embodiments; 

FIG. 43 is a block diagram showing an image pickup 
apparatus according to the embodiment; 

FIG. 44 is a chart for explaining a method of forming 
luminance signals; 

FIG. 45 is a chart for explaining a method of forming 
color signals; 

FIG. 46 is a chart showing a color filter pattern according 
to still another embodiment of the present invention; 

FIG. 47 is a block diagram showing an image pickup 
apparatus according to the above embodiment; 

FIG. 48 is a chart for explaining a method of forming 
luminance signals; and 

FIG. 49 is a chart for explaining a method of fanning 
luminance signals. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

The preferred embodiments of the present invention will 
be described below with reference to the accompanying 
drawings. 

FIG. 4 is a block diagram showing a color image pickup 
apparatus according to the first embodiment of the present 
invention. Referring to FIG. 4, an image pickup element 1 
serves to convert light from an optical low-pass filter SO (to 45 
be described later) into an electrical signal. Four types of 
color filters (to be described later) are arranged in the image 
pickup element 1. 

An imaging signal is read out from the image pickup 
element 1 far each pixel. Each imaging signal read out from 
the image pickup element 1 is subjected to CDS processing 
(removal of reset noise or clock noise from each output 
signal from the image pickup element 1), AGC processing 
(optimization of the input signal level), and the like in a 
preprocessing circuit 2. Thereafter, the signal is A/D- 
converted by an A/D converter 3 at a timing synchronized 
with a read clock. The A/D converter 3 preferably has linear 
characteristics in consideration of subsequent color 
processing, and preferably has the ability to perform A/D 
conversion with a precision of 8 bits or more in consider- 
ation of quantization errors. 

The A/D-converted signal is temporarily written in a 
buffer memory 4 for two-dimensional signal processing to 
be performed later. Signals required for the processing are 
read out from this buffer memory 4. 

Signals corresponding to luminance signals are sequen- 
tially read out from the buffer memory 4 in the order 



corresponding to the pixel array of the image pickup element 
1 so as to be input to a low-pass filter 5. The signals input 
to the low-pass filter 5 undergo a predetermined band 
limiting process. Hie resultant signals are subjected to 
predetermined processes in a 7 conversion section 14 and an 
enhancer 15 to become a luminance signal Y /rf containing a 
high-frequency component. Note that the luminance signal 
Y^containing the high-frequency component may be 
obtained by using a high-pass filter (HPF) 21 as in the 
second einbodiment shown in FIG. 5. 

The luminance signal Y^mcluding the high-frequency 
component, obtained in this manner, is synthesized with a 
low-frequency component Y L (of a luminance signal), 
obtained by a method to be described later, by a luminance 
signal formation circuit 16. The resultant signal is then 
output as a luminance signal Y. 

Meanwhile, signals corresponding to color signals Mg, 
Gr, Cy, and Ye are read out from the buffer memory 4 in the 
same manner as described above and are converted into 
synchronized color signals Mg, Gr, Cy, and Ye through four 
interpolation filters 6, 7, 8, and 9. Subsequently, these four 
types of color signals are input to an RGB conversion 
section 10 to be converted into three primary color signals 
R, G, and B. Note that this conversion is performed by the 
following matrix operation: 

(2) 



If' 




Kg' 
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In this case, the matrix A is a 3x4 matrix which is 
optimized to allow spectral characteristics Mg(X), Gr(X), 
Cy(X), and Ye(X) of the signals Mg, Gr, Cy, and Ye from the 
image pickup element 1 to approximate ideal spectral char- 
acteristics R(A,), G(X), and B(A,), of the signals R, G, and B, 
defined by the NTSC standards. 

The three signals R, G, and B formed by the RGB 
conversion section 10 are converted into signals aR, G, and 
(JB by a white balance section 11, thus achieving a white 
balance. This white balance adjustment may be automati- 
cally performed on the basis of color temperature data 
obtained by using a white balance sensor 22, as shown in 
FIG. 5. 

Subsequently, the R, G, and B signals are y-converted by 
table conversion in a y conversion section 12 and are input 
to a color difference matrix section 13. In the color differ- 
ence matrix section 13, the following color difference con- 
version conforming to the NTSC standards is performed to 
generate the aforementioned low-frequency component Y^ 
and two color difference signals R-Y and B-Y: 
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The color difference signals R-Y and B-Y are subjected to 
a predetermined band limiting process in low-pass filters 18 
and 19, respectively. The resultant signals are then output 
60 The low-frequency component Y L is subjected to a prede- 
termined band limiting process in a low-pass filter 17. The 
low-frequency component is then synthesized with the lumi- 
nance signal Yjcf containing the high-frequency component 
by the luminance signal formation circuit 16, as described 
above, and the resultant signal is output as the luminance 
signal Y. Each of these signals is converted into an analog 
signal by a D/A converter (not shown). 
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This synthesizing operation performed by the luminance 
signal formation circuit 16 is based on the following con- 
version formula: 



where (LPF17) indicates a low-pass filter having the same 
band as that of the low-pass filter 17, and (LPF17)*Y W 
indicates that the same band limiting process as that per- 
formed by the low-pass filter 17 is performed with respect to 
the luminance signal Y H containing the high-frequency 
component 

Such conversion is performed for the following reasons. 
In general, the low-frequency component Y L has proper 
luminance in terms of spectral characteristics but is not 
necessarily sufficiently high in S/N ratio. In contrast to this, 
the luminance signal Y H is sufficiently high in S/N ratio but 
does not necessarily have proper luminance in terms of 
spectral characteristics. 

If the S/N ratio of the low-frequency component Y L is 
sufficiently high, the 6 value may be fixed to "1". In this 
case, the processing performed by the luminance signal 
formation circuit 16 is equivalent to replacing a portion, of 
the luminance signal Y^, having the same band as that of the 
low-frequency component Y L with the low-frequency com- 
ponent Yi,. 

If the spectral characteristics of the luminance signal Y N 
containing the high-frequency component are sufficiently 
close to the proper luminance characteristics, the 5 value 
may be fixed to **0 M . In this case, no processing is performed 
by the luminance signal formation circuit 16, and the lumi- 
nance signal Y H containing the high-frequency component 
is directly output as the luminance signal Y. 

Note that the 5 value may be changed in accordance with 
the brightness and color information of an object to be 
photographed, considering the S/N ratio and color repro- 
duction characteristics. 

FIGS. 6 and 7 arc charts, each showing an embodiment of 
a color filter array arranged in the image pickup element 1 
described above. Each color filter array is constituted by four 
color filters, i.t, Mg, Or, Cy, and Ye filters. In each array, 
color filters of different colors are repeatedly arranged at 2 
Fh in the horizontal direction, and color filters of different 
colors are repeatedly arranged at 2 Pv in the vertical direc- 
tion while they are offset in the horizontal direction by Ph, 
thereby constituting an offset sampling structure. 

Note that the color filter arrays shown in FIGS. 6 and 7 
need not be used as they are. For example, Mg and Cy filters 
or Gr and Ye filters may be interchanged. In addition, filter 
colors are not limited to the aforementioned four colors. For 
example, one of the aforementioned four colors may be 
arbitrary selected, and red Rd, blue Bl, white Wh, and the 
selected color may be combined to form a color filter array. 

FIG. 8 is a graph showing the frequency characteristics of 
the image pickup element 1 having such a color filter 
arrangement, more specifically, a graph showing the posi- 
tions where carrier components are generated near the base 
band. 

According to this graph, luminance carriers are generated 
at the positions of frequencies (±1/Ph,0) in the horizontal 
direction, and color carriers are generated at the positions of 
frequencies (±1/2 Fh, ±1/4 Pv) and (0, ±172 Pv). 

FIGS. 9 and 10 show the frequency characteristics in 
more detail FIGS. 9 and 10 are graphs respectively showing 
the frequency characteristics of the image pickup element 1 
having the color filter arrangements shown in FIGS. 6 and 
7. 

According to these frequency characteristics, color carri- 
ers are generated at neither the position of a frequency (0, 



1/4 Pv) in the vertical direction nor the position of a 
frequency (1/2 Fh, 0) in the horizontal direction. For this 
reason, the occurrence of color moire' due to aliasing from 
these color carriers is suppressed. Furthermore, since no 

5 earner is generated atmer>oeition of a frequency (1/2 Ph,0), 
the resolution can be improved to f^l/2 Ph as a theoretical 
resolution limit 

FIG. 11 shows the schematic arrangement of the optical 
low-pass filter 50 described above. Referring to FIG. 11, the 

10 optical low-pass filter 50 comprises a first optical member 
constituted by a birefringent plate 51 for splitting an incident 
light beam into two light beams separated from each other 
by a first distance dj in a direction inclined at 45° with 
respect to the horizontal direction, a second optical member 

15 constituted by a birefringent plate 52 for splitting an incident 
light beam into two light beams separated from each other 
by a second distance d^ in the horizontal direction, and a 
third optical member constituted by a birefringent plate 53 
for splitting an incident light beam into two light beams 

20 separated from each other by the first distance d x in a 
direction inclined at -45° with respect to the horizontal 
direction. With the effects of the birefringent plates 51 to 53, 
an incident light beam propagates along the paths shown in 
FIG. 12. 

25 The optical low-pass filter 50 is designed such that the 
first distance d x satisfies the conditions indicated by the 
following relation: 

1.6 PkPvAPh an iV ooe 9IS^£Z4 PhPvAPh an 8+2 Pv cob 
^ 9KPSe£tf2) (5) 

If the first distance d 1 is below the lower limit indicated 
by relation (5), aliasing distortion due to color carriers, 
especially the occurrence of color moir6, cannot be sup- 
pressed. In contrast to this, if the first distance d t exceeds the 
35 upper limit, sufficiently high resolution cannot be obtained. 
Therefore, in order to prevent color moire* and obtain suf- 
ficiently high resolution, the first distance dj must satisfy 
relation (5). 

The transfer characteristics (MTF characteristics) of the 
40 optical low-pass filter 50 can be expressed by the following 
equation: 

fliCfe, Jv) = fcos(7id5tfe) • cos (nf NT • di(fe + (6) 
45 A)) * ooaCft/NfT • dt(fg -fr))\ 



SO 



55 



Consider, for example, an image pickup element used for 
an apparatus based on the NTSC scheme and having a 
display screen with an aspect ratio of 3:4, in which element, 
the number of effective pixels is about 640 (in the horizontal 
direction)xabout 480 (in the vertical direction). In such an 
image pickup, the following equation can almost be satis- 
fied: 



Pb=Pv 



In this embodiment, therefore, the first distance d x and the 
second distance dj are defined as follows: 



60 



dy&~m >pk d, =phn 



(8) 



FIG. 13 shows MTF characteristics which can satisfy the 
conditions defined by equations (8). In addition, FIG. 14 
shows characteristics on a spatial frequency plane (f w , f v ). 
65 Referring to FIG. 14, broken lines 54a, 54&, 55a, 556, 56a, 
and 566 represent resonance frequencies which the optical 
low-pass filter 50 trap. It is apparent from this graph that the 
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optical low-pass filter 50 traps all the carrier frequencies for Providing that the base band components of the respective 

both luminance and color difference signals. colors are sufficiently band-limited by two-dimensional fil- 

As is apparent from FIG. 13, since frequency components ters such as interpolation filters 6, 7, 8, and 9, the carrier 

having frequencies higher than the resolution limit fre- components at the position of a frequency (0, 1/2 Pv) in 

quency fr/=l/2 Ph are satisfactorily suppressed, no aliasing 5 FIGS. 9 and 10 are defined as follows: 
distortion of luminance occurs. Therefore, color moire" can 

be reduced. In addition, as is apparent from equations (6), Mg(fc,/v) = OKfo/v) = -W»/v>. < 13 ) 
(7), and (8) and FIG. 13, since the MTF does not decrease 
below -15 dB in the frequency region of f H £l/2 Ph, 

resolution almost equivalent to the resolution limit fre- 10 Mg(fmfr) = Gr(f a >f v y = WH>fv)* 
quency Fh can be ensured. 

In an image pickup element widely used nowadays, in W» fv) = W»/v) = -Wx,M 

which the sampling frequency for a luminance signal is four Therefore, the carrier components of the R, G t and B 

times as high as a carrier frequency f sc for a color difference signals at this point can be defined according to equations 

signal, the following equation can be substantially estab- is ( 2 ), (11), (12), and (13) as follows: 
lished: 

ph=12 pv ^ WmM = *k*n-*i2 + *n + * r WfB>fv)> < 14 ) 

In this case, if the first and second distances dj and d^ are Wwfv) = ±H»« - + h» + «*W3*/vX 

defined as follows: 20 z^/ v ) = ±<^ 3] - a „ + aj3 + a i4 Wf»f v ) 

d^Pk, df=Ph/i (10) According to these equations, in each row of the matrix A, 

the same effects as those described above can be obtained ? the sum of the coefficients of mefet andse^nd cohunns 

As described above, according to this embodiment, resc- 18 ^ tothe coefficients of the third and fourth 

lution as high as the resolution limit can be attained owing 25 columns » 

to the characteristics of the optical low-pass filter 50, while 42 n 4fl n =sa 0 4**(*=i,2,3) (15) 

color moire* can be satisfactorily suppressed owing to the , , . A _ L _ _ JT> . , . 

effect of the color filter array of inc image pickup clement 1 *? cam« components of the R, G, andB signals at the 

and the characteristics of the optical low-pass filter 50. That P° mt <°- elumnated, and hence no color earners 

is, the optical low-pass filter 5« need not be designed in 30 are S^^T*^ . ,, ,* . - . _, 

consideration of the traded between resolution and color Sumta £ ^"f™ (15) U P"*? 

moir6 but can be designed with priority being given to ** P°>*t (0, -1/2 .W) symmetneal with me point (0, W Pv) 

resolution. Therefore, the optical low-pass filter 50 can be ™* t0 *e °ngta are , elmmiated ' ™ d no 00101 

made low in profile and compact, and hence the overall carriers are generated at this point 

image pickup optical system can be made compact. 35 , n Bv Agoing the matrix A in an RCTconversaon section 

The third embodiment will be described below with 10 " to saUsf y condition indicated by equation (15), the 

reference to FIG 15 carrier components of color signals in the vertical direction 

•me same reference numerals in FIG. 15 denote the same «"> 1x5 eUnnnated^ Therefore, color moire caused by an 

constituent elements as In FIG. 4. Since the basic operations achromatic object having a fine pattern canbe reduced.- 

of the constituent elements shown in FIG. 15 are the same 40 u ^1h*t* gunri^r&^bcaca^yprf^ 

as those in FIG. 4, a detailed description thereof will be * gain adjustment circuit 21 1 on Jhc basu ofthe 

omfactl respective color temperature data, white balance adjustment 

This embodiment is characterized in that a gain adjust- in a white balance section 11 need not be perfumed. 

ment circuit 20 is arranged before anA/D converter 3 so that hi this embodiment^ me luminance signal Y and the color 

the gains of imaging signals obtained from pixels Mg, Gr, 45 difference signals R-Y and B-Y may be output according to 

Cy, and Ye areadjusted byme gain adjustment circuit 20 to a non-interlace scheme or an interlace scheme regardless of 

be uniform with respect to an achromatic object Note that Pfsence/absence of the gain adjustment drcuit 20. The 

this gain adjustmentmay be performed with respect to an signals output by the non-interlace scheme are advantageous 

analog signal before A/D conversion, as shown in HG. 15. m ^compression in a card camera and the like and 

If. however, the quantization precision of the A/D converter * are also suitable as signals to be input to a computer The 

3 is sufficiently high, gain adjustment may be performed s ^ out P ut bv * e m *f** 6cheme ™ V "^J? 

-«w a /n „L:„. a video camera and a still video camera. In this case, a D/A 

after A/D conversion. . . „ . L , , . ^ 

With the aforementioned gain adjustment, superposition ™ verte [ * to convert the output signals into 

of unnecessary modulation components on a luminance analog signals. . .,. ... 

signal can be prevented, and a color signal can be improved 55 ^ ^^^^ mterpolatton process, band krniUng 
as follows process, RGB conversion process, and the like are all linear 

Assume that the matrix A in conversion formula (2) is ^f 85 ^ ** ^ r^son even if these processes are 
defined as performed in different orders, the same result can be 

obtained. Therefore, the respective processing sections may 
^=fa</X»= W,; >=1,2,3,4) (11) 60 be arranged in a different order in accordance with hardware 

e- . . . ^ • , ■» * r- r^. used. That is, any circuits which can be made equivalent to 

Smcefce gains of the respective pixels Mg Gr, Cy and Ye ^ ^ above-described block diagram by only 

are adjusted by the gain adjusu^nt orcuit 2« to be umfem order to processing drcuits are induded in the 

with respect to an achromatic object, the base band compo- tin ti 

nente of Mg, Gr Cy, and Ye can be expressed by using a a ™ above, according to the first to third 

luncnon <ki w , i v j as rouows. embodiments, color filters are arranged in the image pickup 

WvWb^Ws/i'Wb/i'Ws/J (12) element 1 in the form of an offset sampling array, and the 
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optical low-pass filter SO is designed to satisfy predeter- 
mined conditions. In addition, a matrix operation to be 
performed when R, G, and B signals are formed from signals 
output from the image pickup element 1 is performed by 
using a 3x4 matrix which is designed such that the sum of 
the coefficients of two arbitrary columns in every row is 
made equal to the sum of the coefficients of the remaining 
columns. With this operation, color carriers generated in the 
horizontal and vertical directions can be reduced, and hence 
color moir6 caused by these color carriers in the horizontal 
and vertical directions can be reduced. Furthermore, since 
color moir6 caused at the resolution limit can be eliminated, 
the resolution can be increased to the resolution limit 

Recently with the introduction of high image-quality 
formats such as S-VHS and high-band 8 m/m formats, the 
image quality of color image pickup apparatuses has been 
rapidly improved. Although the image quality of a camera 
section must be improved in accordance with this 
improvement, a single-plate scheme color image pickup 
apparatus using only one image pickup element has already 
come close to its limit 

FIG. 16 shows an arrangement of this two-plate scheme 
color image pickup apparatus. 

Referring to FIG. 16, light incident on a lens 121 is split 
into two optical paths, and the two tight beams are focused 
on a luminance signal image pickup element 123 and a color 
signal image pickup element 124. 

A signal extracted from the luminance signal image 
pickup element 123 is subjected to a predetermined process 
in a luminance signal processing circuit 125 and is input to 
an encoder 127. A signal extracted from the color signal 
image pickup element 124 is separated into R, G, and B 
signals by a color signal processing circuit 126, and the R, 
G, and B signals are input to the encoder 127. These 
luminance and color signals are converted into NTSC sig- 
nals by the encoder 127 and are extracted by an output 
terminal 128. 

Conventionally, for example, image pickup elements hav- 
ing the color filter arrangements shown in FIGS. 17A and 
17B have been respectively used as the luminance signal 
image pickup element 123 and the color signal image pickup 
element 124. As is apparent from FIGS. 17A and 17B, the 
luminance signal image pickup element 123 has color filters 
of green (G) arranged on the entire surface, and the color 
signal image pickup element 124 has color filters of red (R) 
and blue (B) arranged in the form of vertical stripes. 

In a two-plate scheme color image pickup apparatus, 
since these two image pickups 123 and 124 are separately 
arranged, the characteristics of the respective image pickup 
elements 123 and 124 can be optimized. In addition, a 
two-plate scheme color image pickup apparatus can acquire 
data twice as much as a single-plate scheme color image 
pickup apparatus and hence can obtain high image quality 
which the single-plate scheme color image pickup apparatus 
cannot provide. 

In this two-plate scheme color image pickup apparatus, 
however, the optical system is complicated, and two driving 
circuits are required in correspondence with the two image 
pickup elements 123 and 124. Therefore, for a color image 
pickup apparatus which alms at reductions In size, weight, 
and cost, a conventional two-plate scheme color image 
pickup apparatus is too large, heavy, and expensive. 

Some single-plate scheme color image pickup apparatus 
can attain image quality as high as that of a two-plate 
scheme color image pickup apparatus. 

As shown in FIG. 18, in this image pickup element, the 
total number of horizontal lines is set to be twice a desired 
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number of scanning lines, and the horizontal lines for 
luminance signals, indicated by Y, and the horizontal lines 
for color signals, indicated by R and B, are alternately 
arranged Signals from these two types of lines arc read out 
5 in pairs, and predetermined signal processing is performed. 
Note that each color filter indicated by Y is designed in 
advance such that its spectral characteristics match visual 
sensitivity. 

Even if the above-described two-plate scheme color 
image pickup apparatus or single-plate scheme color image 
pickup apparatus is used, the above-described problems of 
color moire' and insufficient resolution remain unsolved. 
More specifically, both the image pickup elements shown 
l5 in FIGS. 17A to 18 have the frequency characteristics shown 
in FIG. 19A and 1J>B. As is apparent from this graph, color 
carriers are generated at the positions of frequencies (±1/2 
Eh, 0) and (±1/Ph, 0) in the horizontal direction. Therefore, 
color moire* is caused by the effect of these color carriers. In 
2Q addition, a frequency component which should be theoreti- 
cally resolved cannot be resolved. 

The fourth to eighth embodiments far solving the above- 
described problems posed in the case wherein such a single- 
or two-plate scheme color image pickup apparatus is used 
25 will be described below. Note that the fourth and fifth 
embodiments exemplify the single-plate scheme color 
image pickup apparatus, and the sixth to eighth embodi- 
ments exemplify the two-plate scheme color image pickup 
apparatus. The same reference numerals in the drawings 
30 showing the respective embodiments denote the same parts. 
FIG. 20 is a block showing the fourth embodiment 
Referring to FIG. 20, an image pickup element 61 has 
lines twice as many as a desired number of scanning lines, 
and has color filters arranged in the manner shown in FIG. 
35 21. In the image pickup element 61, G filters are arranged on 
odd lines, and R and B filters are alternately arranged on 
even lines. In this case, of the G filters on the odd lines, the 
filters on the same columns as those of the R filters are 
denoted by G 4 , and the filters on the same columns as those 
40 of the B filters are denoted by G 2 . 

Imaging signals output from the image pickup element 61 
are separated into Gj and G 2 signals and R and B signals by 
color separation sections 63 and 64, respectively. Note that 
in this color separation processing, if signals from the odd 
and even lines shown in FIG. 21 are read out in pairs at once, 
the processing can be simplified 

The gains of the respectively separated signals are prop- 
erly adjusted by white balance sections 65 and 66 and are 
^ subjected to y conversion in y conversion sections 67 and 68. 
This white balance processing may be performed only with 
respect to the R and B signals. In this case, white balance 
gain adjustment is not performed with respect to the G x and 
G 2 signals. 

55 In an adder 69, the difference between the G x signal and 
the R signal output from the y conversion sections 67 and 68 
is calcnlatnd to form a difference signal K-G l . In an adder 
70, the difference between the G 2 signal and the B signal 
output from the y conversion sections 67 and 68 is calculated 

a, to form a difference signal B-G 3 . 

These difference signals R-Gj and B-G 2 are input to a 
color difference signal processing circuit 72 and are con- 
verted into color difference signals R-Y and B-Y according 
to the following conversion formula. Subsequently, color 

62 difference signals R-Y and B-Y having desired frequency 
bands are generated by a low-pass filter (not shown) and the 
like. 
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[r-yi r 0.70 -o.u if*" 01 ] (16) 
B-Yj'l -030 0.89 J [fl- ft J 

Assume that an achromatic object at a position (1/2 Ph, 0) 
on a spatial frequency plane is imaged by the image pickup 
element 61. This object has vertical stripes of a period 2 Ph. 
For such an object, 

R=Qi and B=0 2 

Therefore, both the difference signals R-Gj and B-G 2 output 
from the adders 69 and 70 become zero. 

For this reason, the color difference signals R-Y and B-Y 
output from the color difference signal processing circuit 72 
become zero and hence are not output This means that the 
color carrier at the position of the frequency (1/2 Fh, 0) is 
eliminated. In other words, at the frequency (1/2 Fh, 0), the 
carrier of the R signal and the carrier of the Gi signal are in 
phase, and the carrier of the B signal and the carrier of the 
G 2 signal are in phase. Since the carriers of these difference 
signals R-Gi and B-G 2 at this frequency can be eliminated, 
no color carders are generated. 

The difference signals R-G 1 and B-G 3 output from the 
adders 69 and 70 are also input to a luminance correction 
signal formation circuit 73. The luminance correction signal 
formation circuit 73 is constituted by a constant multiplier 
and an adder for the difference signals R-G t and B-G 2 . That 
is, in the linninance correction signal formation circuit 73, a 
luminance correction signal S is formed from the difference 
signals R-G x and B-G 2 according to the following conver- 
sion formula: 

^(R^ytCifrGJ (17) 

where C x and C 2 are constants. For example, C a =0.30 and 

The G t and G 2 signals output from the y conversion 
section 67 are input to a luminance signal processing circuit 
71 and undergo proper processes, e.g., low-pass filtering and 
enhancement The luminance correction signal S formed in 
the above-described manner is added to- the O t and G 2 
signals by an adder 74. By adding the luminance correction 
signal S to the G x and G 2 signals in this manner, the spectral 
characteristics of the luminance signal are corrected, and a 
proper luminance signal matching visual sensitivity can be 
obtained. 

As described above, according to this embodiment, since 
the color carriers at the frequency f^=l/2 Ph can be 
eliminated, the generation amount of color moh*6 is small, 
thereby obtaining a high-resolution color image pickup 
apparatus. 

The fifth embodiment will be described next 

FIG. 22 is a block diagram showing the arrangement of 
the fifth embodiment. 

Referring to FIG. 22, an image pickup element 62 has 
lines twice as many as a desired number of scanning lines, 
and has color filters arranged in the manner shown in FIG. 
23. In the image pickup element 62, Y filters whose spectral 
characteristics are designed in advance to match visual 
sensitivity are arranged on odd lines, and R and B filters are 
alternately arranged on even lines. In this case, of the Y 
filters on the odd lines, the filters on the same columns as 
those of the R filters on the even lines are denoted by Y lT and 
the filters on the same columns as those of the B filters are 
denoted by Y 2 . 

In mis embodiment, similar to the fourth embodiment 
described above, difference signals R-Y t and B-Y 2 are 
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formed by adders 69 and 70 on the basis of the Y 1 and Y 2 
signals and the R andB signals separated by color separation 
sections 75 and 64. Hie difference signals formed in this 
case are equivalent to the color difference signal R-Y and 

5 B-Y, conversion to color difference signals, indicated by 
conversion formula (16), need not be performed in a color 
difference signal processing circuit 76. In addition, since the 
Y 1 and Y 2 signals which are adjusted to match visual 
sensitivity in advance are input to a luminance signal 
processing circuit 71, the formation of the luminance cor- 

10 rection signal S, indicated by conversion formula (17), need 
not be performed. Therefore, in comparison with the fourth 
embodiment, the processes in the fifth embodiment can be 
simplified. 

In the fifth embodiment having the above-described 
15 arrangement, similar to the fourth embodiment, since the 
color carriers at the frequency f^=l/2 Ph can be eliminated, 
the generation amount of color moire* is small Hence, a 
high-resolution color image pickup apparatus can be 
obtained. 

20 The sixth embodiment will be described next 

FIG. 24 is a block diagram showing the arrangement of 
the sixth embodiment 

Referring to FIG. 24, image pickup elements 81 and 82 
have color filter arrangements shown in FIGS. 2SAand 25B, 

25 respectively. More specifically, in the image pickup element 
81, the G filters far luminance signals are arranged on the 
entire surface. In the image pickup element 82, the R and B 
filters for color signals are alternately arranged in the form 
of vertical stripes. In this case, of the G filters of the image 

30 pickup element 81, the filters at positions equivalent to the 
positions of the R filters of the image pickup element 82 are 
denoted by G lt and the filters at positions equivalent to the 
positions of the B filters are denoted by G 2 . 
This embodiment is characterized by comprising the two 

35 image pickup elements 81 and 82, but is the same as the 
fourth embodiment shown in FIG. 20 in other respects. The 
operations of the respective portions are the same as those in 
the fourth embodiment In this embodiment, therefore, the 
color carriers at the frequency f f/ =l/2 Fh can be eliminated, 

40 and the generation amount of color moir6 is small. Hence, a 
high-resolution color image pickup apparatus can be 
obtained. 

The seventh embodiment will be described next 

FIG. 26 is a block diagram showing the arrangement of 
45 the seventh embodiment 

Referring to FIG. 26, image pickup elements 83 and 84 
have the color filter arrangements shown in FIGS. 27A and 
27B, respectively. More specifically, in the image pickup 
element 83, the G filters for luminance signals are arranged 
50 on the entire surface. In the image pickup element 84, the R 
and B filters for color signals are arranged for every two 
lines in the form of horizontal stripes. In this case, of the G 
filters of the image pickup element 83, the filters at positions 
equivalent to the positions of the R filters of the image 
55 pickup element 84 on an imaging plane are denoted by G lt 
and the filters at positions equivalent to the positions of the 
B filters are denoted by G 2 . 

In the image pickup elements having such filter 
arrangements, color carriers are generated at the positions 
60 indicated in FIGS. 28A and 28B. 

In each of the image pickup elements 83 and 84, scanning 
is performed for every other horizontal line according to an 
interlace scheme, and the G x and G 2 signals and the R and 
B signals are respectively read out from the image pickup 
65 elements 83 and 84. These signals are subjected to prede- 
termined processes in white balance sections 65 and 66 and 
y conversion sections 67 and 68. The resultant signals are 
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then separated into the d and G 2 signals and the R and B 
signals by 1H delay circuits 85 and 86, respectively. 

Difference signals R-Gj and B-G 2 axe formed by adders 
69 and 70 on the basis of the G x and R signals and the G 2 
and B signals separated in this manner. Subsequently, color 
difference signals R-Y and B-Y are formed by a color 
difference signal processing circuit 72. In addition, desired 
band limiting processing is performed by a low-pass filter 
(not shown) and the like. 

Assume that an achromatic object at the position of a 
frequency (0, 1/4 Pv) is imaged by the image pickup 
elements 83 and 84. For such an object, the following 
equations can be established: 

R=G, airiB^ 

Consequently, the difference signals R-G x and B-G 2 become 
zero. That is, the color difference signals R-Y and B-Y 
become zero and hence are not output 

This means that the color carriers at the position of the 
frequency (0, 1/4 Pv) are eliminated. In other words, at the 
frequency (0, 1/4 Pv), the carrier of the R signal and the 
carrier of the G, signal are in phase, and the carrier of the 
B signal and the carrier of the G 3 signal are in phase. 
Therefore, the difference signals R-G x and B-G 3 at this 
frequency can be eliminated. This prevents the generation of 
color carriers. 

The processes performed by a luminance signal process- 
ing circuit 71, a luminance correction signal formation 
circuit 73, and an adder 74 are the same as those in the fourth 
embodiment described above. With these processes, the 
spectral characteristics of a luminance signal are corrected to 
obtain a proper luminance signal matching visual sensitivity. 

When the image pickup elements 83 and 84 are to be 
scanned by the non-intedace scheme, the filter arrangement 
of the image pickup element 84 may be designed, such that 
the R and B filter groups are alternately arranged in the 
respective lines in the form of horizontal stripes, as shown 
in FIG. 28B. As shown in FIG. 28A, in this case, the 
arrangement of the O x and G 2 filters of the image pickup 
element 83 is also determined in correspondence with the 
filter arrangement of the image pickup element 84. Accord- 
ing to mis arrangement, color carriers arc generated at the 
positions indicated by FIGS. 29A and 29B. 

In this case, similar to the case wherein signals axe read 
out by the above-described interlace scheme, carriers at the 
position of a frequency (0, 1/2 Pv) can be eliminated, thus 
obtaining an excellent image. Furthermore, if a memory 
capable of storing signals corresponding to one frame is 
used, an interlaced image can be also obtained. 

The image pickup elements 83 and 84 may have the filter 
arrangements shown in KEGS. 30A and 30B, respectively. In 
this case, in the image pickup element 84, the R and B filters 
are arranged in an offset manner, as shown in FIG. 30B. In 
accordance with mis arrangement, the Gand G 2 filters of the 
image pickup element 83 are also arranged in an offset 
manner, as shown in FIG. 30A. According to these 
arrangements, color carriers are generated at the positions 
indicated by FIGS. 31A and 3DB. 

In this case, by performing the same processing as mat 
described above, the color carriers at the positions of fre- 
quencies (±1/2 Ph, 1/2 Pv) can be eliminated to obtain an 
image having high quality. 

In mis embodiment, the difference signals R-Gj and B-G 2 
are switched at a one-horizontal signal period (1H). 
Therefore, in order to obtain desired color difference signals 
and luminance correction signal S by the conversion pro- 
cessing in the color difference signal processing circuit 72, 
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indicated by equation (16), and the formation processing of 
the luminance correction signal S in the luminance correc- 
tion signal formation circuit 73, indicated by equation (17), 
for example, input signals to the respective circuits must be 

s switched at a period of 1H. 

The eighth embodiment will be described next 
FIG. 32 is a block diagram showing the arrangement of 
the eighth embodiment 
Referring to FIG. 32, image pickup elements 87 and 88 

10 have the color filter arrangements shown in FIGS. 33A and 
33B, respectively. More specifically, in the image pickup 
element 87, the Y filters for luminance signals are arranged 
on the entire surface. In the image pickup element 88, the R 
and B filters for color signals are alternately arranged in the 

15 form of vertical stripes. In this case, of the G filters of the 
image pickup element 87, the filters at positions equivalent 
to those of the R filter groups of the image pickup element 
88 on the imaging plane are denoted by Y lf and the filters at 
positions equivalent to those of the B filter groups are 

20 denoted by Y 2 . 

As described above, this embodiment is characterized by 
comprising the two image pickup elements 87 and 88, but is 
the same as the fifth embodiment shown in FIG. 22 in other 
respects. The operations of the respective components are 

25 the same as those in the fifth embodiment That is, in the 
eighth embodiment, the color carriers at the frequency f = l/2 
Ph can be eliminated. Therefore, there is provided a high- 
resolution color image pickup apparatus in which the gen- 
eration amount of color moh*6 is small. In addition, in the 

30 eighth embodiment, the luminance signal formation process- 
ing can be simplified as compared with the sixth and seventh 
embodiments. 

In the eighth embodiment the color filter arrangement 
shown in FIG. 33B is used for the image pickup element 88. 

35 However, the present invention is not limited to this, and any 
one of the color filter arrangements shown in FIGS. 27B, 
28B, and 30B may be used. In mis case, separation process- 
ing of the Y x and Y 2 in the image pickup element 87 may be 
performed in the same manner as the separation processing 

40 of the Gj and G 3 signals in FIGS. 27A, 28A, and 30A, 
respectively. 

In the fourth to eighth embodiments described above, an 
excellent image having a small amount of color moird and 
high resolution can be obtained both in a motion picture 
45 image pickup apparatus such as a video camera and in a still 
picture image pickup apparatus such as an electronic still 
camera. 

As described above, according to the embodiments of the 
present invention, a color filter array arranged in an image 

50 pickup element has an offset sampling structure in which 
color filters of difference colors are repeatedly arranged at a 
two-pixel period in the horizontal direction, and color filters 
of different colors are repeatedly arranged at a two-pixel 
period in the vertical direction while being offset by one 

55 pixel in the horizontal direction. With this arrangement, 
color carriers generated in the horizontal and vertical direc- 
tions can be reduced and color moire" caused by these color 
carriers in the horizontal and vertical direction can be 
reduced. Furthermore, since color moire* caused at the reso- 

60 lution limit can be eliminated, the resolution can be 
increased to the resolution limit 

In addition, an optical low-pass filter is constituted by an 
optical member for splitting an incident light beam into two 
light beams separated from each other by the distance D, 

65 which satisfies the predetermined conditions, in a direction 
inclined clockwise or counterclockwise at the angle 9 with 
respect to the scanning direction of the image pickup ele- 
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racnt Therefore, satisfactory MTF characteristics can be 
obtain in a region lower than the resolution limit, and the 
resolution can be increased to the resolution limit. 
Furthermore, color moire* caused by aliasing distortion of 
color carriers generated at frequencies higher than that of the 5 
resolution limit can be reduced. 

That is, an optical low-pass filter need not be designed in 
consideration of the trade-off between resolution and color 
moire* but can be designed with priority being given to 
resolution. Therefore, a high-resolution still frame image 10 
can be obtained. 

The gain of an output from an image pickup element is 
adjusted such that outputs from four types of color filters are 
made constant In addition, a matrix operation to be per- 
formed when the R, G, and B signals are formed from these is 
outputs is performed by using a 3x4 matrix designed such 
that the sum of the coefficients of two arbitrary columns of 
every row is jnadp. equal to the sum of the coefficients of the 
remaining two columns. For this reason, the color carriers 
generated in the vertical direction can be further reduced, 20 
and color moire* in the vertical direction can be further 
reduced, thereby obtaining an excellent image. 

If the color image pickup apparatus of the present inven- 
tion is a multiple-plate scheme color image pickup apparatus 
using a plurality of image pickup elements or a single-plate 25 
scheme color image pickup apparatus using an image pickup 
element having horizontal lines twice as many as those of a 
general image pickup element, first to third color filters of 
different colors are arranged for the respective pixels. With 
this arrangement, two types of difference signals are formed 30 
on the basis of signals obtained from given pixels for which 
the second and third color filters are respectively arranged, 
and signals obtained from pixels which are at positions 
equivalent to those of the given pixels on the imaging plane 
and for which the first color filters are arranged. Color 35 
difference signals are obtained on the basis of these differ- 
ence signals. Therefore, the color carriers generated in the 
horizontal and vertical directions can be reduced, thus 
reducing color moire' and obtaining a hig^-resolution image. 

In addition, since a luminance signal is formed by per- 40 
forming proper correction with respect to a signal obtained 
from each pixel for which the first color filter is arranged, 
using these difference signals, an image having a high S/N 
ratio can be obtained. As the first color filter, a luminance 
signal color filter which is designed in advance in consid- 45 
eration of the aforementioned correction is used, and a 
luminance signal is formed on the basis of a signal obtained 
from this color filter. Therefore, an image having high 
quality can be obtained, and moreover, the luminance signal 
formation processing can be simplified. so 

Since these processes are performed digitally, there are 
merits in terms of the uniformity and stability of perfor- 
mance. 

Another embodiment of the present invention will be 
described next. 35 

FIG. 34 shows the color filter arrangement of an image 
pickup element according to this embodiment This color 
filter array is constituted by the Mg, G, Cy, and Ye filters, 
which are arranged at periods 2 Ph and 2 Pv in the horizontal 
and vertical directions, respectively, thus f coning a square 60 
sampling structure. The color filter array need not be 
designed to be exactly the same as that shown in FIG. 34. 
For example, the lines of Mg and G and the lines of Cy and 
Ye may be interchanged, and the columns of Mg, Cy, G, . . 

. and the columns of G, Ye, Mg, may be interchanged. 65 

FIG. 35 shows the spatial frequency characteristics of this 
image pickup element. Referring to FIG. 35, the arrows 
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indicate carriers of the respective colors, and their directions 
indicate phase relationships. As is apparent from this graph, 
no color carriers are generated at a frequency (0, L/4 Pv) in 
the vertical direction. Although color carriers are generated 
at a frequency (172 Ph, 0) in the horizontal direction, since 
an independent read operation is performed, the color car- 
riers can be reduced by gain adjustment and by using a 
primary color separation matrix. These color carriers can 
also be reduced by a low-pass filter. 

An arrangement of a color image pickup apparatus of the 
embodiment will be described next. FIG. 36 is a block 
diagram showing the embodiment An image pickup ele- 
ment having the four types of color filters shown in FIG. 34 
and arranged on an element surface is arranged in a sensor 
1. Image signals are read out from the sensor 1 in units of 
pixels. The image signals read out from the sensor 1 in units 
of pixels are subjected to processes such as CDS and AGO 
processes in a preprocessing circuit 2. Thereafter, the signals 
are A/D-converted by an A/D converter 3 at a timing 
synchronized with a read dock. The A/D converter 3 pref- 
erably has linear characteristics in consideration of subse- 
quent color processing, and preferably has the ability to 
perform A/D conversion with a precision of 8 bits or more 
in consideration of quantization errors. 

The A/D-converted signals are temporarily written in a 
buffer memory 4 and are read out therefrom for two- 
dimensional signal processing to be performed later. 

Luminance signals are read out from a buffer memory 4 
in the order corresponding to the pixel array of the image 
pickup element The readout signals are subjected to a 
predetermined band limiting process in a low-pass filter 5. 
The resultant signals are converted into a luminance signal 
Y H containing a high-frequency component ttirough a yO 
conversion section 14 and an enhancer 15. The luminance 
signal Y H is then synthesized with a luminance low- 
frequency component Y £ , obtained by a method to be 
described later, by a luminance signal formation circuit 16. 
The resultant signal is output as a luminance signal Y. 

Meanwhile, signals corresponding to color signals Mg, 
Gr, Cy, and Ye arc read out from the buffer memory 4 and 
are input to four interpolation filters 6, 7, 8, and 9 to be 
converted into synchronized color signals Mg, Gr, Cy, and 
Ye. Subsequently, these color signals are input to an RGB 
conversion section 10 to be converted into three primary 
color signals R, G, and B. 

In a white balance section 11, the R, G, andB signals are 
converted into signals ocR, G, and f*B, thus achieving a white 
balance. 

In a y conversion section 12, y conversion of the R, G, and 
B signals is performed by table conversion. 

In a color difference matrix section 13, conversion con- 
forming to the NTSC standards is performed to generate the 
aforementioned low-frequency component Y^ and two color 
difference signals R- Y and B-Y. The color difference signals 
R-Y and B-Y are subjected to a predetermined band limiting 
process in low-pass filters 18 and 19, respectively. The 
resultant signals are then output 

The low-frequency component Y L is subjected to a pre- 
determined band limiting process in a low-pass filter 17. The 
low-frequency component is then synthesized with the lumi- 
nance signal Y H containing the high-frequency component 
by the luminance signal formation circuit 16, as described 
above, and the resultant signal is output as the luminance 
signal Y. This. processing is performed by the following 
conversion formula: 

j^y fl ^(ij , Fi7)«y ff +^y Il (0S5Si) (4) 

where (LPF17) indicates a low-pass filter having the same 
band as that of the low-pass filter 17, and (LPF17)* Y H 
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indicates that the same band limiting process as that per- then, the carrier components of the R, G, andB signals at the 

formed by the low-pass filter 17 is performed with respect to point (0, 1/2 Pv) are eliminated, and hence no color carriers 

the luminance signal Y H . Such conversion is performed far are generated. That is, when conversion formula (2) used by 

the following reasons. In general, the low-frequency com- the RGB conversion section 10 satisfies the conditions 

ponent represents proper luminance in terms of spectral 5 indicated by equation (15), the carrier components of color 

characteristics but is not sufficiently high in S/N ratio. In signals in the vertical direction can be eliminated. Therefore, 

contrast to this, the luminance signal Y H is sufficiently high color moire* caused by an achromatic object having a fine 

in S/N ratio but does not necessarily represent proper pattern can be reduced. 

luminance in terms of spectral characteristics. If the S/N In the horizontal direction, the carrier components at the 

ratio of the low-frequency component is sufficiently high, 10 position (1/2 Pv, 0) in FIG. 35 are defined as: 
the 6 value may be fixed to "1". In this case, the processing 

is equivalent to replacing a portion, of the luminance signal ^iifsfv)=Cy(fa/ i/H(Wv). Oif H / v)=^(fn/ v) (20) 

Y H , having the same band as mat of the low-frequency Therefore, the carrier components of the R, G, and B signals 

component Y L with the low-frequency component Y^ If the M?6bstm represented according to equations (2), (11), 

spectral characteristics of the luminance signal Y H are 15 and (20) as follows: 
sufficiently close to the proper luminance characteristics, the 

5 value may be fixed to "0". In this case, no processing is R(fwfy) = ^ + flfj _ Qjj + flj#Wai/i ,x m 
performed by the luminance signal formation circuit 16, and 

the luminance signal Y w is directly output as the luminance <%> f v ) = + a n - + a M Wfm fv\ 

signal Y In addition, the 6 value may be adaptiYely changed 20 t . . w . 

in accordance with the brightness and color information of B(f » fr) = ™ J + a »~ a » + fl ~*^/v) 

an object to be photographed, considering S/N ratio and In this case, in each row of the matrix A, if the sum of the 

color reproduction characteristics. coefficients of the first and third columns is equal to the sum 

FIG. 37 shows still another embodiment of the present of the coefficients of the second and fourth columns, the 

invention. In this arrangement, the gains of pixels Mg, Gr, 25 carrier components of the R, G, and B signals at the point 

Cy, and Ye are adjusted by a gain adjustment circuit 20 to be ( 1/2 Ph, 0) are eliminated, and hence no color carriers are 

uniform with respect to an achromatic object This gain generated at this point By satisfying these conditions, the 

adjustment may be performed with respect to an analog carrier components of color signals in the horizontal direc- 

si glial before A/D conversion, as shown in FIG. 37. If, tion can be eliminated. Therefore, color moire caused by an 

however, the quantization precision of an A/D converter 3 is 30 achromatic object having a fine pattern can be reduced, 

sufficiently high, gain adjustment may be performed after When priority is to be given to the vertical resolution, the 

A/D conversion. With this processing, superposition of matrix A is defined according to equations (19). When 

unnecessary modulation components on a luminance signal priority is to be given to the horizontal resolution, a matrix 

can be prevented, and a color signal can be improved as is designed according to equations (21). Since no color 

follows. 35 moire 1 in oblique directions is caused at ( 1/4 Ph, 1/4 Pv), the 

Assume that in conversion formula (2) used by the RGB color moire" in the horizontal and vertical directions can be 

conversion section 10, conveniently eliminated by optimally switching or adjusting 

matrixes. By adaptively perfecting this adjustment in 

A=(a#x*=iA3;/=iA3,4) (u) accordance with an image to be picked up, an excellent 

„. ^ . ^. . . XM „ - 40 image having high resolution can be obtained. 

T^/^ i? C ^ V \^ ^ Note that if gain adjustment can be accurately performed 

are adjusted by the gain ad^stmen ^rcint20 tobe uniform ^ ^ ^ circuit 20 in aceoroance with the 

with reject to an achromatic obje£ the base band compo- ^^^lox ternperarures, no adjustment is required in 

ncnts of Mg, Gr, Cy, and Ye can be expressed by using a me F white ^cT^tion 11. 

function WhJv} as follows: 43 ^ each of the above-described embodiments of the 

Mtf^^^yWfxteWBfvWfdfv) (12) F"L sem invention, the luminance signal Y and the color 

difference signals R-Y and B-Y may be output by a non- 
Providing that the base band components of the respective interlace scheme or an interlace scheme. The signals output 
colors are sufficiently band-limited, the carrier components by the non-interlace scheme are advantageous in image data 
at the position of a frequency (0, 1/2 Pv) are defined as 50 compression in a card camera and are also suitable as signals 
follows : to be input to a computer. The signals output by the interlace 

scheme can be used for a video camera or a still video 

Mg^^ v y^^ y y^^ y \Cy<f^ v y^Yeij^yy^^ v ) (18) camera. In this case, a D/A converter for converting the 

~- , . ^ ... _ ^ . _ . , cutout signals into analog signals is required at the output 

Therefore, the carrier components of the R, G, and B signals 5J terminal.^ 

I^J^St^J^ aCCa " fing to eqUad ° DS (2) ' FIG. 38 shows still another embodiment of the present 

invention. 

Hie four types of color filters shown in FIG. 6 are 
arranged in a sensor 1. Signals simultaneously read out from 

Gtfmfv) = - 623 + + a»Wff>M 60 the sensor 1, two pixels at a time, are added together. The 

resultant signal is read out as an image signal. A read 
^p/v) = <-^7 - + + a^yx/v/v) operation is performed according to the non-interlace 
In this case, in each row of the matrix A, if the sum of the scheme as follows. First, signals from two lines are added to 
coefficients of the first and second columns is equal to the obtam signals from one line. The read line is shifted down- 
sum of the coefficients of the third and fourth columns, i.e., 63 ward by one line, and signals from two pixels, i.e., upper and 

lower pixels, are added together. Image signals read out 

a a -n2a=a a Hi^(f=i^) (15) from the sensor 1 are subjected to processes such as CDS 



(12), and (18) as follows: 

AOs* A) = <r*n - an + + *u)MfB»fv\ (&) 
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and AGC processes in a preprocessing circuit Z Hie result- 
ant signals are A/D-converted by an A/D converter 3 at a 
timing synchronized with a read clock. 

The A/D- converted, image signals are stored in four or 
. more lines in a line memory 4. A write/read operation with 
respect to the line memory 4 is performed as follows. 
Signals from two pixels are simultaneously read out while 
the read line is shifted line by line, as shown in FIG. 39, and 
the readout signals are treated as a signal from one pixel. 
Signals from one line are formed in this manner, and signals 
from four lines are recorded. The signals from the first line 
are extracted from the line memory 4, and the extracted 
signals and the signals from the current line ((N+4)th line) 
are averaged to form pixels. When the read line is an odd 
line, signals corresponding to the read line and signals from 
a line immediately preceding the read line are simulta- 
neously read out If the read line is an even line, signals from 
the read line and a line three lines ahead of the read line are 
simultaneously read out 



10 



13 



converter 3, as shown in FIG. 41. If the quantization 
precision of A/D conversion is sufficiently high, the gain 
adjustment circuit 130 may be arranged behind the A/D 
converter 3. 

According to each embodiment described above, there is 
provided a color image pickup apparatus which can reduce 
color raoird in the horizontal and vertical directions, has high 
resolution, and can form a still frame image. In addition, an 
optical low-pass filter need not be designed in consideration 
of the trade-off between resolution and color moir6 but can 
be designed with priority being given to resolution. For this 
reason, the optical low-pass filter can be made low in profile 
and compact. Therefore, the overall image pickup apparatus 
can be made compact 

Since all the processes are performed as digital processes, 
there are merits in terms of the uniformity and stability of 
performance. 

Still another embodiment will be described next 
This embodiment is designed to provide an image pickup 
apparatus having higher resolution. This apparatus includes 



A luminance signal is read in different manners depending 20 an image pickup element having a plurality of pixels 



on whether the read line is an even line or an odd line. When 
the read line is an odd line, the sum of signals from a line 
three lines ahead of the read line is calculated along the time 
base. When the read line is an even line, the sum of signals 
corresponding to a line immediately preceding the read line 
is calculated along the time base. For example, letting Yn be 
the luminance signal to be read out, Yn =Cy+G+Ye+Mg= 
2R+3G+2B for an odd line, and Yn=Mg+Cy4G+Ye=2R+ 
3G+2B for an even line. That is, Yn=2R+3G+2B regardless 



arranged in the form of a matrix, with color filters of the 
same color being arranged for the respective pixels in the 
horizontal direction, and color filters of three or more 
different colors being alternately arranged for the respective 
25 pixels in the vertical direction. 

In still another embodiment, an image pickup apparatus 
includes an image pickup element having a plurality of 
pixels offset in the horizontal direction for every other lines, 
with color filters of the same color being arranged for the 



of whether the read line is an even line or an odd line. This 30 respective pixels in the horizontal direction, and color filters 



luminance signal Yn has a value very close to the spectral 
characteristics defined by the NTSC standards, and hence is 
directly used as a luminance signal Y. This luminance signal 
is band-limited by a low-pass filter 51 to be output as the 
luminance signal Y through an enhancer 50 and a y conver- 
sion section 14. 

Color difference signals 2R-G and 2B-G are formed on 
the basis of signals from two lines, extracted in the manner 
shown in FIG. 40, by using a primary color separation 



of three or more different colors being alternately arranged 
for the respective pixels in the vertical direction. 

According to the embodiment described above, the color 
filters are arranged in the form of horizontal stripes. With 
35 this arrangement, since a luminance signal is formed by 
adding signals from three pixels in the vertical direction, 
high resolution can be attained. 

In addition, the color filters are arranged in the form of 
horizontal stripes in a solid-state sensor having an offset 



matrix 60 according to S1-S2 and S2-S1 where SI is the 40 arrangement With this arrangement, since a luminance 



signal is formed by using signals from three pixels in the 
vertical direction, high resolution can be attained 

Each embodiment will be described below with reference 
to the accompanying drawings. 

As shown in FIG. 42, three color filters, Le., yellow (Ye), 
green (G)» and cyan (Cy) filters, are repeatedly arranged, on 
a solid-state sensor, in the horizontal direction in the form of 
horizontal stripes. 
Referring to FIG. 43, signals output from the sensor 1 are 



target pixel and S2 is the pixel shifted along the time base 
by one pixel, thereby performing primary color separation. 
For example, for an even line, S2-^Sl=(Yc+MgHCy+G>= 
2R-G, and Sl-S2=(Mg4<VHG+Ye)=2B-G. In addition, 
since Yn-(S1+S2)=5G, the R, G, and B signals are extracted 45 
from this. The R, G, and B signals, which have undergone 
primary color separation, are converted into signals OR, G, 
and (SB by a white balance section 11, thus achieving a white 
balance. 

These signals are then subjected to y conversion in a Y so input to an A/D converter 7 to be converted into digital 
conversion section 12. signals. Signals from one horizontal line are stored in a line 

In a linear matrix section 90, conversion conforming to memory circuit 8 using three 1H line memories. Signals 
the NTSC standards is performed according to equation (4) from a total of three lines, ic, the first, second, and third 
to extract color difference signals R-Y and B-Y. The signals lines, are sequentially stored in the line memory circuit 8. 
are subjected to a predetermined band limiting process in a 55 Since three colors are repeatedly arranged, signals are 
low-pass filter 100. The resultant signals are then output always stored, as combinations of Ye, G, and Cy, in the 
FIG. 43 shows still another embodiment of the present memory corresponding to three horizontal lines. On the 
invention. In this embocament, the gains of signals indepen- basis of the digital data corresponding to three horizontal 
dently read from Mg, G, Cy, and Ye pixels are adjusted by lines, a luminance signal is generated by a Y signal genera- 
a gain adjustment circuit 130 to be uniform with respect to 60 tion circuit 9, and a color signal is generated by a subtraction 
an achromatic object to be photographed, and signals from 
two pixels are added by an adder 140 and are stored in a line 
memory 4. Since signals from two pixels are independently 
read and added by the adder 140 and are stored in the line 



circuit 10. The generated signals are respectively output, as 
an analog luminance signal Y and an analog color signal C, 
through D/A converters 11 and 12. 

A method of generating the luminance signal Y will be 



memory 4, the gains of signals from the respective pixels 65 described next with reference to FIG. 44. Aluminance signal 



can be made uniform, thereby suppressing color moire\ The 
gain adjustment circuit 130 may be arranged before an A/D 



Y„ corresponding the nth line is obtained by adding the 
following signals in the vertical direction: 
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Ye corresponding to the (n-l)th line 
G caxrespoading to the nth line 
Cy corresponding to the (n+l)tb line 
That is, 

Y=Y<*<HCy 

Similarly a luminance signal Y^ corresponding to the 
(n+l)th line is generated as follows: 



Yu = Yiu Yn* Yv* Yu, Y-& • . • 

{CZYen +<h\ + (7n + 20^6}, 
{(Yen* Ye u + 2Gn + Cyn+ CyuW}, . * . 

Similarly, a horizontal luminance signal of the third 
line is represented as follows: 



10 



15 



The scheme of this embodiment is characterized in that no 
line step is caused because the component ratio of the 
respective color signals, i.e., Cy+G+Ye, of the luminance 
signal Y Generated from each line is constant 

A method of generating color signals will be described 20 
next with reference to FIG. 45. The R signal is obtained by 
subtracting G, which is located closest to Ye, from Ye. That 
is, 



ft« = Yih lazt Y&, Yto . . . 

i(Ou + Cu + 2Cyn + Yen + Ye2&6} f 
{<2Gu + Cyn + Cyn + 27^6}, . . . 



R=Y*-C 



25 



Similarly, 



As described above, at any lines, luminance signals are 
generated with a constant ratio, i.e., 

Y=Ye+G*Cy 

Therefore, no line step is caused A method of generating 
color signals will be described next 

First, band limiting is performed by the LFF 13 with 
respect to the following Ye signal string of the first line: 

Ye^YenyYe^lb^Ye^Ye^ 



B=Cy~G 



Similarly, band limiting is performed by the LPF 13 with 
respect to the G signal string of the second line and the Cy 
30 signal string of the third line: 



35 



Although both the R and B signals cannot be obtained 
with respect to all the horizontal lines, synchronization may 
be performed by using a line memory or the like. 

In the above-described embodiment, the Cy, G, and Ye 
filters are combined. It is, however, apparent that other color 
filters may be combined. 

Still another embodiment will be described next with 
reference to the accompanying drawings. 

As shown in FIG. 46, three color filters, i.e., the Ye, G, and 40 following equations: 
Cy niters, are repeatedly arranged on a solid-state sensor in 
the horizontal direction. Although the color niters in FIG. 46 
are arranged in the same order as mat of the color niters in 
FIG. 42, the positions of the respective pixels in HG. 46 are 
shifted in the horizontal direction for every other lines, 

Referring to FIG. 47, signals output from a sensor 1 are 
converted into digital signals by an A/D converter 7, and 
signals corresponding to one horizontal line are stored in a 



By performing subtraction processing, in the vertical 
direction, with respect to low-frequency color signal strings 
Yc Lln ,Q Lln9 and Cy Lln obtained in this manner, a red signal 
(R^) and a blue signal (B^) are obtained. 
This subtraction processing is performed according to the 
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Although both the R and B signals cannot be obtained 
with respect to all the horizontal lines, synchronization may 
be performed by using a line memory or the like. 

In the above-described embodiment, a calculation method 
equivalent to interpolation is used to generate a luminance 



line memory of a line memory circuit 8. Signals from a total 50 signal. However, as shown in FIG. 49, a luminance signal 



of three lines, Le., the first, second, and third lines, are 
sequentially stored in the line memory circuit 8. Since three 
colors are repeatedly arranged, signals are always stored, as 
combinations of Ye, G, and Cy, in the memory correspond- 
ing to three horizontal lines. On the basis of the digital data 55 
corresponding to three horizontal lines, a luminance signal 
is generated by a Y signal generation circuit 9, and a color 
signal C is generated through a low-pass filter (LPF) 13 (to 
be described later) and a subtraction circuit 10. The gener- 
ated signals are respectively output, as an analog luminance 
signal Y and an analog color signal C, through D/A con- 
verters 11 and 12. 

A method of generating luminance signals will be 
described next with reference to FIG. 48. For example, a 65 horizontal direction, and niters of three or more different 
horizontal luminance signal Y^ of the second line is rep- colors being alternately arranged for the respective pixels in 
resented as follows: the vertical direction, thereby providing an image pickup 



60 



may be generated by adding signals from three pixels Ye, G, 
and Cy. In mis case, the luminance signal is represented as 
follows: 

Ylm ~ *21» Ym YOf *24» . • . 

{(Ye a + Gn + Cy l2 y3} f ... 

As described above, the above-described embodiment 
includes an image pickup element having a plurality of 
pixels arranged in the farm of a matrix, with color filters of 
the same color being arranged for the respective pixels in the 
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apparatus which can attain high horizontal resolution with a 
simple arrangement If, for example, a sensor having 780 
pixels in the horizontal direction according to the NTSC 
scheme is used, a horizontal resolution of about 780x 
0.8=600 can be expected. In addition, since no inconve- 5 
niences such as line steps are caused, uniform, high image 
quality can be obtained. 

According to still another embodiment, an image pickup 
apparatus includes an image pickup element having a plu- 
rality of pixels offset in the horizontal direction for every 10 
other lines, with color filters of the same color being 
arranged for the respective pixels in the horizontal direction, 
and color filters of three or mare different colors being 
alternately arranged for the respective pixels in the vertical 
direction. With this image pickup element, high horizontal is 
resolution can be obtained, which is a characteristic feature 
of an offset arrangement of pixels, and uniform image 
quality free from line steps can be realized with a simple 
arrangement. 

What is claimed is: 20 
1. A color image pickup apparatus comprising an image 
pickup element having a plurality of pixels two- 
dimensionally arranged in horizontal and vertical directions, 
wherein horizontal lines twice as many as a number of 
horizontal lines required to obtain a desired image are 25 
arranged in said image pickup element, 
the horizontal lines comprising first and second horizontal 
lines which are alternately arranged such that first color 



filters are arranged on each first horizontal line and 
second and third color filters are arranged for every 
other pixel on each second horizontal line, 
a first difference signal being formed by calculating a 
difference between a signal output from a given pixel 
for which the second color filter is arranged and a signal 
output from a pixel which is on a same column as the 
given pixel and for which the first color filter is 
arranged, 

a second difference signal being formed by calculating a 
difference between a signal output from a given pixel 
for which the third color filter is arranged and a signal 
from a pixel which is on the same column as the given 
pixel and for which the first color filter is arranged, and 

color difference signals being formed on the basis of the 
first and second difference signals, 

wherein a luminance signal is formed by synthesizing a 
signal output from a pixel for which the first color filter 
is arranged and the first and second difference signals. 

2. An apparatus according to claim 1, wherein the first 
color filter corresponds to a color green. 

3. An apparatus according to claim 1, wherein the first 
color filter corresponds to a color which has a transparent 
luminance component. 

4. An apparatus according to claim 1, wherein the second 
and third color filters correspond to primary color filters. 
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